1. Field of the Invention
The present invention relates to the field of data entry in computer systems.
2. Background Art
A typical computer system consists of a central processing unit (CPU), main memory such as random access memory (RAM), a data entry device, including a positioning device, a mass storage device such as one or more disk drives, a display and/or a printer. In the prior art, the data entry device often consists of a keyboard, on which a user enters data by typing. The positioning device of a prior art computer system may consist of a "mouse" or other cursor positioning device.
Computer systems have also been developed that are directed to handwritten data entry rather than keyboard data entry. These systems are often characterized by the use of a pen, stylus, or other writing device, to enter handwritten data directly on the display of the computer system. Alternatively, these systems may provide for a user to enter data on a digitizing tablet or other input device, with the image of the written input displayed on a separate computer display output device. The writing device for entering handwritten or freestyle stroke input information is not limited to a pen or stylus, but may be any input device such as a mouse, trackball, pointer, or even a person's fingers. Such systems are not necessarily limited to receiving data generated by human users; for example, machine generated data may also be inputted and accepted to such systems.
One class of this handwriting entry computer system that receives handwritten data input is referred to as "pen based" computers. In a pen based computer system, a writer can input information on a display by "writing" directly on the display. A writing device, such as a pen or stylus, is used to enter information on the display. In a typical pen-based computer system, a user touches the stylus to the display and writes as he would on a piece of paper, by making a series of pen strokes to form letters and words. A line appears on the display that follows the path of travel of the pen point, so that the pen strokes appear on the display as ink would appear on a handwritten page. Thus, the user can enter information into the computer by writing on the display. In addition to handwriting, special functions based on input strokes are supported on pen based computer systems. For example, a writer may strike certain previously entered handwritten input by making the strokes of an "X" over the entry. These special functional strokes or actions are referred to herein as "gestures". Pen based computers typically have a display surface that serves as both an input receiving device and as an output display device.
One problem in a handwriting entry computer system is determining the beginning and ends of characters and words. This process is referred to as "splitting." Some prior art systems rely on the proximity of a pen or stylus to the display or entry means as a cue to indicate the beginning and ends of characters and/or words. However, users may not always lift the pen out of proximity from the display or entry means when writing a new character or new word. Incorrect splitting decisions can lead to errors in display of handwritten or translated text since separations between words and characters may not be recognized.
The display of a pen-based entry or other handwriting entry computer system may have bordered regions called "fields" that define locations or areas where data is to be entered, or that are associated with particular actions. For example, if the software application provides a data entry display that is an information form with fields for "First Name", "Last Name", "Age", "Occupation", etc., certain information is generally associated with each field. These locations are referred to here as "objects" or "fields". Each object has a boundary associated with it. When data is entered, the computer must determine which object is to be associated with the writing or gesture. The object intended to be associated with the data input is called the target object. The process of identifying the target object is referred to here as targeting. The method and apparatus that performs the function of targeting is referred to here as a targeting system.
It is the nature of handwriting that the writer often will write strokes or characters outside of defined boundaries of objects or fields. Where the target object is relatively small compared to typical handwriting paper guidelines, for example, the writer will typically find it difficult to stay within the lines when writing into that object. Resultant writing strokes may fall partly within the boundaries of the target object and partly in areas of the display that are outside of the boundaries of the target object and/or in areas that do not contain any objects. In addition, writing strokes may fall partly within the boundaries of objects adjacent to the target object.
There are a number of prior art schemes for targeting. These prior art targeting schemes will typically provide incorrect targeting decisions when applied to handwritten input. A common characteristic of these prior art schemes is that targeting decisions about the target field are made at the beginning of the writing operation, for example, at the time of initial contact of the writing entry device with the display or input device. These prior art schemes typically result in a substantially degraded targeting decision.
In the following description, the prior art schemes will be described in connection with a handwriting data entry computer system that uses a pen or stylus as the data entry device and where data is entered by writing directly on the display. This is for purposes of example only, and is not intended to limit the description.
In one prior art targeting scheme, the target is defined as the object touched by the pen upon initial contact. Thus, in this prior art scheme, the target object is the object, (if any), that is associated with the point of initial contact of the stylus, pen or analogous input device. This scheme is described in connection with the examples of FIGS. 1 and 2.
FIG. 1 illustrates an example of a writing entry computer system that includes a housing 101, a display 102 and a writing tool, or "pen" 103. The display 102, for example, shows as display output, the outline of a form for data entry. This form can be generated, for example, by an application program running on the pen based computer system. The form may include a number of objects or fields. For example, in the example of FIG. 1, the form includes the "First Name" field 104, the "Last Name" field 105, the "Company" field 106 and the "Date" field 107.
In this example, fields 104-107 are arranged such that the lower boundary of one field is coincident with the upper boundary of the adjacent field. As a result, if a user entering data in one field writes outside the boundaries of that field, the stray writing may fall within the boundaries of another adjacent field. Consider the example of entering the name "Tom" in the "First Name" field 104, the name "Jones" in the "Last Name" field 105, and the name "International" in the "Company" field 106. When an entry begins with a capital letter such as "T", "J", or "I", it has been found that a user is likely to begin the pen stroke that defines the letter "J" near the top of the target field. The first stroke of the letter will be entered by the user beginning in the field above the target field, or by beginning the first stroke entirely outside any field, as shown in the examples of FIG. 1.
For example, in FIG. 1, the top stroke of the letter "T" of the name "Tom" in field 104 extends outside of the boundaries of field 104 and a portion of the top stroke of the letter "T" is not located within the boundaries of any field. The letter "J" of the name "Jones" in field 105 has a portion of its top stroke that is not within the boundaries of any field. The top stroke of the letter "J" crosses a corner of field 104 before extending into field 105. The "I" of the name "International" is found partially within the boundaries of field 105 and extends within the boundaries of field 106.
It has been found that the letter "T" is typically written with two strokes, a horizontal "cap" stroke, and a vertical stroke. The letter "J" is also typically written with two strokes, namely a horizontal stroke for the top or cap of the "J", and a curved stroke for the bottom hook of the letter "J". The letter "I" is can be written with a single vertical stroke, or, alternatively, with horizontal top and bottom strokes connected by a single vertical stroke. In the examples of FIGS. 1 and 2, the "I" is written as a single vertical stroke.
The first letters of each of the words in fields 104-106 respectively are illustrated in detail in FIG. 2. Referring to FIG. 2, an enlarged view of handwritten strokes 201 and 202 that together represent the letter "T" is illustrated. The beginning point of top stroke 201, that is, the point where the stylus first makes contact with the display, is at point 206. The end point of stroke 201, that is, the point where the stylus is lifted from the display, is at point 207. The beginning point of bottom stroke 202 is at point 208. The end point of stroke 202 is at point 209. Note that the beginning point of bottom stroke 202 lies on stroke 201.
Still referring to FIG. 2, the letter "J" is shown formed by top stroke 203 and bottom stroke 204. The beginning of top stroke 203 is at location 211. The end point of top stroke 203 of the "J" is at location 212. The beginning of bottom stroke 204 is at point 213 (which also lies on stroke 203). The end point of bottom stroke 204 is at point 214.
The letter "I" is formed by a single stroke 205 with a beginning point at location 215. The end point of stroke 205 is at location 216.
The intended target fields for the names "Tom", "Jones", and "International" are fields 104-106 respectively. The application of the prior art scheme that defines the target field as the first point of contact of the stylus results in incorrect targeting assignments for each name. Referring to the letter "T", the first point of pen contact with the display, namely point 206, does not lie in any field at all. Therefore, using the prior art scheme describing a target field cannot be properly determined, resulting in an error. Similarly, the first point of contact of the top stroke 203 of the letter "J", namely point 211, does not fall within the boundaries of any field. Therefore, no target field is defined by this prior art scheme, also resulting in error.
Referring to the stroke 205 that defines the letter "I" of the name "International", the first point of contact of the stylus is at point 215, located within the boundaries of field 105. Thus, the prior art scheme defines the target object as field 105. However, the intended target field is field 106. Thus, there is a targeting error.
Another prior art targeting scheme defines the target object as the first object touched by a pen stroke of the writer, regardless of where the first point of contact of the stylus may occur. That is, after the pen touches the display and the writer begins to make a stroke, the very first object or field touched by the stroke is considered to be the target. The operation of this scheme is also illustrated by referring to the characters of FIG. 2.
Referring to the "T" of the name "Tom", the first point of contact of the stylus on the display is at point 206. The first field contacted by the stroke 201 is field 104. Therefore, the second prior art scheme makes the correct targeting decision for this character, namely field 104. However, the second prior art scheme renders incorrect decisions for the other characters of FIG. 2.
Referring to the letter "J" of the name "Jones", the first point of contact of stroke 203 is at point 211, outside the boundaries of any field. The first field contacted by stroke 203 is field 104. Therefore, the second prior are scheme incorrectly identifies the target field as field 104, instead of field 105. This is an incorrect decision.
Referring to the letter "I" of the name "International", the first point of contact of stroke 205 is at point 215, within the boundaries of field 105. Therefore, using the prior art targeting scheme, the target field is incorrectly identified as field 105, when the intended target field is field 106.
Thus, it is a disadvantage of prior art targeting schemes that they do not make accurate targeting decisions when data is entered outside the boundaries of the intended target field.
There are a number of patents that are directed to or relate to computer systems that operate on handwritten data.
Fox, U.S. Pat. No. 4,727,588, provides a system to create and edit handwritten images of documents on an all points addressable digitalizing tablet. The device of Fox stores pen stroke coordinates in order to form word extremities. By comparing a current pen stroke with the preceding pen stroke and the associated word extremity, a new word can be detected. Since each word has an extremity associated with it, the system is able to format a handwritten document. In Fox, a word extremity consists of a rectangular area and a zone of proximity extending from each side of the rectangle. Each new pen stroke is compared with the current zone of proximity of the current word. If any part of the current pen stroke falls within the zone, it is part of the current word. If the current stroke is outside the zone of proximity, the beginning of a new word is detected. Thus, Fox employs a rectangle to help indicate the beginning of a new word. Fox does not teach a rectangle/area overlap technique to provide targeting
Ward, U.S. Pat. No. 4,562,304, is directed to emulating keyboard input using a hand-print terminal that includes a digitizing tablet. In Ward, characters are written with a pen in boxes of a grid paper that is disposed upon the digitizing tablet. The characters are converted to ASCII and stored in a line buffer memory at their respective column positions. The contents of the line buffer is serially output to the computer just as a keyboard supplies serial data to a computer. The system of Ward allows only one row of data to be "worked on" at a time. Therefore, the beginning of a new row is determined by comparing the current character row position on the grid with that of the previous character. If the row values differ, the line buffer memory is cleared and a new row can be entered and edited. Ward teaches writing characters within a specified box to allow for character recognition. As such, Ward does not teach a targeting scheme for characters written in adjacent boxes or written only partially in one box.
Tsugei, U.S. Pat. No. 4,656,317, is directed to an apparatus for providing handwritten data to computer memory by categories. The invention uses a format paper with predefined "write areas" disposed upon a digitalizing tablet wherein each "write area" is a category of information to be processed. After information is written within the "write areas" with a pen, it is stored in RAM by category. The invention of Tsugei uses a computer program to recognize when information has been written within a specified area on a digitizing tablet. Tsugei does not disclose a targeting technique in the case where information is written outside the designated areas.
A system that allows for pen input for different application programs such as a word processor or database is described in Sklarew, U.S. Pat. No. 4,972,496. The input pen is used to designate a location to enter data. An input box or line appears on the screen and the pen is used to input text. The invention of Sklarew provides a "learning" feature in which handwritten characters are matched to a database to determine the characters. A character not found in the database is appended. Thus, the invention is adaptable to all handwriting styles. Sklarew provides for handwritten data input within a rectangular input box and does not utilize a targeting scheme.
The invention in Hardin, U.S. Pat. No. 4,817,034, is directed to a system for replicating handwritten signatures. A pen is used to input a signature that is stored in computer memory for future use. The signature data can be provided to a plotter to replicate the handwritten signature. To enter a signature, an input rectangle and signature line is provided by software. Hardin utilizes a rectangular input area for pen input. The system of Hardin does not disclose a method for targeting a signature that exceeds an input area bounds.
A device for editing text is disclosed in van Raamsdonk, U.S. Pat. No. 4,475,239. As the user enters text via a keyboard, a printing head prints the characters on a record carrier which is disposed upon a digitalizing panel. As the characters are entered, they are also stored in computer memory. After a page is stored in memory and printed, a pen is used to assist in editing. For example, the user places alignment marks on the record carrier to allow the system to align the printed text with the corresponding text in memory. Next, using the pen, the user marks the location on the record carrier to perform an operation, such as insert text. After the location is marked, the keyboard is used to enter new characters. The keyed-in characters are inserted into the computer's memory and do not appear on the record carrier. Van Raamsdonk teaches the use of alignment marks to define an edit area. However, van Raamsdonk does not disclose a targeting scheme for marks written outside the edit area.
Sato, U.S. Pat. No. 4,947,156, discloses a handwritten character input device that includes a means to correct deviations on a display due to variations in angles of the input pen. The invention accounts for and corrects LCD dots that are inadvertently lit because of the angle that the input pen makes with the display screen. Sato relates generally to the field of pen based computer systems and does not disclose a targeting scheme.
The following is a patent teaching a scheme that uses a rectangle overlap scheme in a conventional keyboard based computer system.
Ikegami, U.S. Pat. No. 4,677,585 teaches a system in which information having a predetermined format is stored as a card image within a data processing system. The user defines a card by providing coordinates to indicate the size of fields and the desired location on the card. In the case of character data, the fields may be more than one row, thus forming a rectangle. The invention disclosed in Ikegami allows the user to define rectangular areas in which to input character data. The user also defines the location on a card that the rectangular input area will occupy on the display screen. The invention allows for more than one character field may be placed on a card. Therefore, the system utilizes a rectangle overlap technique to determine whether any adjacent character fields intersect. However, the scheme of Ikegami is carried out before any character data entry is performed. That is, the rectangle areas are defined prior to any data input.